PowerScale: OneFS: Service Principal Names for Kerberos Authentication

Introduction
A service principal name (SPN) is the name by which a client uniquely identifies an instance of a service. The SPN is unique, even for multiple service instances on computers within an Active Directory environment. If multiple names are used by clients for authentication purposes, a service instance is assigned multiple SPNs. This includes one unique SPN for each SmartConnect zone name or alias in DNS. For more information, see the Service Principal Names article in the Microsoft Developer Network Center.

Kerberos with NFS in a Non-Active Directory Environment
OneFS does not automatically set up and configure NFS Kerberos with a non-Active Directory authentication server (such as MIT or HEIMDAL). This requires a different manual configuration for the non-Active Directory environment listed in the following article. See OneFS: How to configure the Isilon Cluster to use Kerberos with NFS in a non-Active Directory environment, KB Article 16584.

Kerberos with Active Directory Environment
In order to connect to a cluster by name without providing a username and password in an Active Directory-based environment, Kerberos authentication must be used. 

When accessing a cluster using Kerberos, the client establishes a Kerberos ticket with the cluster based on the DNS name the client is using to connect. When joining an AD domain, the cluster registers any SmartConnect zone names or aliases as SPNs configured on the cluster machine account on the domain. Any additional SmartConnect zone names or aliases created after joining the cluster to the domain must have SPNs manually added. This is true for both the DNS name and short name. 

There must also be a delegation record on the DNS server for each new SmartConnect zone name or alias.

SPNs must be unique across an Active Directory forest. If duplicate SPNs or machine accounts exist, authentication failures may occur. See OneFS: How to find duplicate Service Principal Names (SPNs) in Active Directory, https://www.dell.com/dci/fp/session/authorize?client_id=3a4eea6a-4a4e-4f2e-be4a-adc5d51a357a&redirect_uri=https%3A%2F%2Fwww.dell.com%2Fsupport%2Fkbdoc%2Fen-us%2F000032723

While joining a cluster to multiple AD domains, existing SmartConnect zones at the time of joining adds SPNs for each AD provider. This may result in duplicate SPNs registered between the two domains. This causes problems (for FQDN SPNs) if the two domains share a two-way trust (Microsoft Trusts) with clients connecting to the cluster in a cross-realm fashion. A client from DomainA accessing an Access Zone on the cluster using DomainB uses incorrect machine account to encrypt the ticket, resulting in "KRB5KRB_AP_ERR_MODIFIED" errors.

To view registered SPNs:

OneFS 6.5 and earlier versions:

# isi auth ads spn list

OneFS 7.0 and later versions:

# isi auth ads spn list --domain=<FullyQualifiedDomainName>

The output appears similar to the example below:

# isi auth ads spn list --domain=MY.DOMAIN.COM
SPNs registered for isicluster$:
     HOST/isicluster
     HOST/isicluster.MY.DOMAIN.COM

Symptoms of missing SPNs

• OneFS creates an event/alert if SPNs are missing when connected to Active Directory similar to those below (KB 502666):

  Recurring: AD server missing needed SPN(s) HOST/isicluster, HOST/isicluster.isilon.com; try 'isi auth ads spn check'
  
  AD server missing needed SPN(s) HOST/zonecifs.isilon.com, HOST/zonemgmt.isilon.com, HOST/nfs.isilon.com; try 'isi auth ads spn check'

• Authentication to the cluster by SmartConnect zone name or short name fails, but connecting using IP works.
• High amount of NTLM authentication requests to domain controllers.

To find the missing SPN:

OneFS 6.0 and earlier versions:
List the SmartConnect zones on the cluster:

# isi networks list pools

Then compare to the list of registered SPN:

# isi auth ads spn list.

SPNs registered for isicluster$:
     HOST/isicluster
     HOST/isicluster.MY.DOMAIN.CORP

 
OneFS 6.5 versions: 

# isi auth ads spn check

OneFS 7.x versions:

# isi auth ads spn check --domain=<FQDN>

OneFS 8.x and later versions:

# isi auth ads spn check --provider-name
or
# isi auth ads spn check <provider-name>

Note: In OneFS, multiple domains and access zones can be configured. The command may have to be run for each domain joined to the cluster. It is recommended that the domain be in capital letters as some earlier versions are case-sensitive. Multiple access zones with trusted domains can also cause duplicate SPN issues. Contact support for assistance with multiple access zones with trusted domains.

Creating or adding SPNs require a user account with administrative rights to the domain.

IMPORTANT!
User must specify an AD administrator username when adding the SPNs. If the user fail to do so, the user receives the following error:

LdapError: Failed to modify attribute[19]

Use only the username itself unqualified (no domain designation).

To add missing SPN using repair option in OneFS 6.5 and later versions:

OneFS 6.5 versions:

# isi auth ads spn check --repair --user=<domainadmin>

OneFS 7.x:

# isi auth ads spn check --repair --user=<domainadmin> --domain=<FQDN>

OneFS 8.x:

# isi auth ads spn fix <providername> --user=<admin user of AD>

Note: In OneFS, multiple domains and access zones can be configured. Using the repair option with multiple access zones and trusted domain may cause duplicate SPNs to occur. It is recommended that the SPN be created using the "isi auth ads spn create" command.

Warning: Some customer environments may involve multiple clusters using the same SmartConnect zone names. These may rely on failover automation to add/remove SPNs between clusters, and altering DNS for failover purposes. Running the fix/repair commands may introduce authentication issues if done, use with discretion. Isilon recommends only adding SPNs that are needed on a per SPN basis instead.


To add a single SPN in OneFS 6.5 and later versions:

OneFS 6.5 versions:

# isi auth ads spn create --spn=<service>/<DNS name> --user=<domainadmin>

OneFS 7.x:

# isi auth ads spn create --spn=<service>/<DNS name> --user=<domainadmin> --domain=<FQDN>

OneFS 8.x:

# isi auth ads spn create <providername> --spn=<service>/<DNS name> --user=<admin user of AD>

Note: Additional SPNs may be added if needed (for example, if CNAMEs are in use) beyond the "missing SPNs" list.

Related Articles:

"Authentication services can fail if the Service Principal Name (SPN) is incorrect or missing," 89649
"OneFS: How to create SPN accounts to allow Kerberos authentication using SmartConnect DNS entries," 16528
"How to view an SPN list in a Microsoft Active Directory environment," 16589
"SQL client cannot "Bulk Insert" files from an Isilon cluster to a SQL database," 89574
"How to enable Mac OS X single sign-on (SSO) to Active Directory-enabled CIFS shares in OneFS 5.5.x - 6.5.x," 16675
"Isilon OneFS 7.1.0.0: SMB2 clients cannot connect to the cluster using Kerberos authentication," 174024
"OneFS: How to find duplicate Service Principal Names (SPNs) in Active Directory," 186215.
"OneFS: How to configure the Isilon Cluster to use Kerberos with NFS in a non-Active Directory environment," 16584.
"OneFS: AD server missing needed SPNs alert," 502666